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Why  MRLs? 


“Advanced  weapon  systems  cost  too  much,  take  too  long  to  field, 
and  are  too  expensive  to  sustain”  -  Congress,  OSD,  CSAF,  GAO 


•  Production/manufacturing  processes  are  major  contributor 

-  Recent  GAO  study  of  core  set  of  26  programs:  RDT&E  costs  up  by  42% 
and  schedule  slipped  by  20% 

•  $42. 7 B  total  cost  growth 

•  2.5  years  average  schedule  slip 

-  Characteristics  of  successful  programs: 

•  Mature  technologies,  stable  designs,  production  processes  in  control 

•  S&T  organization  responsible  for  maturing  technologies,  rather  than 
program  or  product  development  manager 

•  Need  way  to  mitigate  impact  of  diminishing  manufacturing 
infrastructure 

-  People,  policy,  programs  gutted 

-  Lost  recipe  on  how  to  manage  manufacturing  risk 

-  Won’t  get  infrastructure  back  but  still  need  to  manage  manufacturing  risk 
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Technology  Readiness  Levels 

(TRLs) 


Provide  a  common  language  and  widely-understood 
standard  for: 

•  Assessing  the  performance  maturity  of  a  technology  and  plans 
for  its  future  maturation 

•  Understanding  the  level  of  performance  risk  in  trying  to 
transition  the  technology  into  a  weapon  system  application 

TRLs  leave  major  transition  questions  unanswered: 

•  Is  the  technology  producible?  Reproducible? 

•  What  will  these  cost  in  production? 

•  Can  these  be  made  in  a  production  environment? 

•  Are  key  materials  and  components  available? 


•  Common  language  and  standard  for 

-  Assessing  the  manufacturing  maturity  of  a  technology  or 
product  and  plans  for  its  future  maturation 

-  Understanding  the  level  of  manufacturing  risk  in  trying  to 
produce  a  weapon  system  or  transition  the  technology  into  a 
weapon  system  application 

•  Designed  to  complement  TRLs 

•  Designed  to  help  set  the  agenda  for  manufacturing  risk 

mitigation 

•  Usage 

-  Army,  for  Future  Combat  Systems  development  efforts 

-  Missile  Defense  Agency  using  EMRLs  on  all  development 
programs 

-  Several  defense  primes  using  on  weapon  system  programs 

-  Mandated  by  AFRL  for  phase-in  on  all  hardware  ATDs 


MRL  Relationships 


Relationship  to  System  Acquisition  Milestones 
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MRL  Evaluation  Criteria 
(Threads) 


•  Technology  and  Industrial  Base 

•  Design 

•  Materials 

•  Cost  and  Funding 

•  Process  Capability  and  Control 

•  Quality  Management 

•  Manufacturing  Personnel 

•  Facilities 

•  Manufacturing  Management 
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MRL  Evaluation  Criteria 
(Threads) 


Acq  Phase 

Pre  CR 

C  R  -  M  S  A  — T  D 

MSB  — 

Criteria 

M  e trie 

MRL  1-3 

M  RL  4 

M  RL  5 

M  RL  6 

Technical 

Technical 

TRLs  1-3 

Must  be  assessed  at  minimum  of 
TRL  4. 

Must  be  assessed  at  minimum  of 
TRL  5. 

Must  be  assessed  at  minimum  of 
TRL  6. 

Technology  & 

Industrial  Base 

TECHNOLOGY 
TRANSITION  TO 
PRODUCTION) 

Identify  technology  leadership  as 
potential  sources 
(foreign/dom  estic); 

(com  m  ercia  I/govern  m  ent) 

IB  capabilities  and  gaps/risks 
identified  for  key  technologies. 

Industrial  Base  analysis 
accomplished  to  identify  potential 
sources. 

IB  capability  in  place  to  support 
mfg  of  developm ent  articles.  IB 
exists  for  similar  components  or 
plan  developed  for  developing 
facilities. 

Design 

Producibility 

Program 

Initial  producibility  assessment  of 
design  com pleted. 

Initial  producibility  of  technology 
completed  (components). 

Initial  producibility  of  technology 
com  pleted  (system  s  level).  Initial 
trade  studies  conducted  - 
performance  vs.  producibility. 

Form ,  Fit,  & 

Function 

Initial  Form,  Fit,  &  Function 
constraints  identified  and 
allocated. 

Form,  Fit,  &  Function  constraints 
identified  and  allocated  at 
com  ponent  level. 

Form,  Fit,  &  Function  constraints 
identified  and  allocated  at  sytem 
level. 

Unique  Components 

Unique  components  identified. 

Unique  component  issues 
identified. 

Plans  completed  to  address 
unique  com  ponent  issues. 

Key  Characteristics 

Key  Performance  Parameters 
(KPPs)  identified. 

KPPs  allocated  at  the  component 
level.  Initial  evaluation  of  Key 
Characteristics  (KC) 
accom  plished . 

Tolorances  established  for  KC. 

Materials 

M  atu  rity 

Characterize  basic  materials  for 
m  anufacturability 

Completed  survey  to  determine  if 
materials  have  been  used  before 
in  a  mfg  environment. 

Preliminary  plans  in  place  to 
address  gaps. 

Related  material  development 
efforts  known.  Maturity  has  been 
assessed  on  similar  materials. 

Maturity  has  been  assessed  on 
sim ilar  m aterials  in  production. 
Specific  programs  identified. 
Preliminary  materail 
specifications  in  place. 

Availability 

New  material  scale-up  challenges 
assessed 

All  exotic/critical/  hazardous 
materials,  and  associated  lead 
times  have  been  identified. 
Significant  material  risks  have 
been  identified:  high  cost, 
availability,  safety,  health, 
hazards,  etc. 

Identify  availability  issues. 

Complete  a  plan  to  address 
availability  issues.  Identify  long 
lead  item s. 

Sources 

ID  sole  source/single 
source/foreign  source  vendors. 

Begin  planning  to  minimizes 
sole/single/foreign  sources. 

Complete  a  plan  that  minimizes 
sole/single/foreign  sources.  Need 
for  Sole/Single/  Foreign  source 
justified.  Identify  potential 
alternative  sources. 

Special  Handling 

Identify  potential  special  handling 
concerns  (i.e.  shelf  life,  security, 
HMMP,  HAZMAT,  storage 
environment,  etc.)  Prepare  MSDS 
as  necessary. 

Identify  special  handling  reqts  (i.e. 
shelf  life,  H  M  M  P,SECU  RITY, 
HAZMAT,  storage  environment, 
etc.)  Review  and  update  MSDS. 

Special  handling  gaps  identified. 

Complete  a  plan  to  address 
special  handling  gaps. 
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MRL  definitions/threads 


•  Rewritten  by  MRL  working  group  in  July  07 

-  Based  on  MRA  experience  to  date 

•  Sponsored  MRL  workshop  on  6-7  Sept  07 

-  Government  and  Industry  representatives 

-  Scrub  definitions/threads 

-  AF  Goal  -  easily  used  by  S&T  and  Acquisition 
communities 

•  New  definitions/threads  to  be  published  soon 
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MRL  Implementation  Approach 


In  partnership  with  Joint  Defense  Manufacturing 
Technology  Panel  (JDMTP) 

-  Develop  and  MRL  definitions  &  policy  language 

•  Conduct  pilots  on  various  programs 

-  Advanced  Technology  Demonstrators 

-  Weapon  System  Acquisition  programs 

-  Demonstrate  benefits  of  using  MRL 

•  Conduct  training  for  key  program  personnel 

-  What  are  MRLs,  how  to  conduct  an  MRA 

•  Air  Force  ManTech  personnel 

•  ATD  and  ACAT  pilot  program  personnel 

•  Various  training  materials  that  can  be  tailored 

-  Transition  to  DAU  once  MRLs  are  in  policy 

•  Put  MRLs  into  policy 

-  AFRL,  AFMC,  AF,  OSD 
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Manufacturing  Readiness  Level  Implementation 

Approach  (ATDs) 


INTRODUCE 


Meet  with  PM  to  get  buy-in  and  gather  program  info 
Customize  MRL  approach  for  program 


TRAIN 


•  Hardware-intensive 

•  Critical  mass  of  time  to  complete 
Newly  developed  products 


•  Train  program  IPT  on  manufacturing  tools  to 
support  manufacturing  maturity  efforts 


OBJECTIVE 

STATEMENT 

DEFINED 


Define  objective  of  program 

Define  what  is  to  be  assessed  and  why 


ASSESS 


u 


Determine  current  MRL 

Develop  plan,  actions,  and  estimate  costs  to  get  to  target  MRL 
Schedule  for  implementation 


INCORPORATE 


MANAGE 


•  Incorporate  MRL  into 
program  baseline 


•  Manage  overall  process 

•  Manage  risk  identification  and  reduction  process 

•  Manage  manufacturing  maturity  to  target  MRL 

•  Reassess  as  appropriate  10 


MRA  Deliverables 


•  Identification  of  current  MRL 

•  Identification  of  key  factors  where  manufacturing 
readiness  falls  short  of  target  MRL 

-  Define  driving  issues 

-  Define  high  risk  areas 

•  Identify  programs  and  plans  to  reach  target  MRL 

-  Generate  the  manufacturing  maturity  plan  (MMP) 

•  Assess  type  and  significance  of  risk  to  cost, 
schedule  and/or  performance 
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Emerging  MRA  Successes 


High  Durability  Hot  Exhaust 

Structures 

"  Provided  identification  of  high  risk 
processes  and  single  point  failures  driving 
scale-up  from  MRL  3 

■  Maturation  plan  provides  awareness  of 
issues  relating  to  move  to  new  production 
facility 

"  Follow-on  MRA  at  new  facility  will  help 
ensure  transition  success 


F135 

■  Enabling  opportunity  to  accelerate 
transition  for  FI  35  thrust 
improvement  by  ~4  years 

”  Advanced  feature  high  cost  driver: 
must  overcome  producibility  issues 

”  Developed  plan  to  mature  from  MRL  3 
to  5  leveraging  commercial  and 
military  IR&D,  F135  program,  and 
ManTech  funding 
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ACAT  MRA  Pilot 


•  Translate  the  successful  MRL  ATD  process  to  acquisition 
programs 

•  Common  themes 

-  Utilize  approximately  the  same  process 

-  Utilize  current  MRL  definitions  to  assess  against 

-  3-5  people  per  MRA 

•  What  is  different 

-  ATDs  focusing  on  MRL  3  -  MRL  6 

•  Assessing  technical  maturity  with  a  goal  of  transition/implementation 

-  ACATs  focusing  on  MRL  4  -  MRL  9 

•  Schedule,  cost,  manning  considerations 

•  Milestone  decisions 

•  Production  planning  process 

•  Will  require  a  more  rigorous  approach 

•  Develop  and  document  a  structured  ACAT  assessment  approach 

-  MRA  Deskbook 

•  First  draft  completed  Mar  07  based  on  ATD  and  limited  ACAT  experience 

•  Drafted  with  SAF/AQRE,  MRL  Working  Group,  and  ASC/EN 

•  Test  drive  on  Reaper 

-  Update  based  on  lessons  learned 
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Manufacturing  Readiness  Implementation 

Approach  (ACATs) 


AMRAAM 


•  Manufacturing  Readiness  Assessment  and 
process  improvements  of  AIM-120  C-7 

•  What:  Performed  a  system-level  MRA  on  the  AMRAAM  C-7 
variant 

-  Looked  at  all  test  and  assembly  steps,  including  FACO 

-  Fourteen  key  suppliers;  over  thirty-five  technology  areas 
examined 

•  Impact:  Based  on  independent  assessment,  AMRAAM  Group 
received  go-ahead  to  proceed  to  next  production  lot  for  C-7 
variant;  reduced  testing  cycle  time  in  particular  cell  by  90% 
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MRL/MRA  Training 


•  Air  Force  ManTech  Personnel 

-  Industrial  Preparedness 

-  MRL/MRA  methodology 

-  Continuing  education 

-  Subject  Matter  Experts 

•  Six  sigma 

•  Lean 

•  ATD  IPTs 

-  MRL  definitions  and  MRAs 

-  Training  can  be  tailored  for  various  audiences 

•  Air  Force  Product  Centers 

-  Based  on  ATD  training  with  lessons  learned  from  ACAT 
experience 

-  DAU 

•  Currently  in  PQM  201 ,  SYS  302,  and  PQM  301 
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Policy  Formulation  Status 


•  DoD 

-  Policy  language  written  by  MRL  working  group 

-  OSD  motivated  to  get  policy  in  place  by  end  of  Sept  07 

-  Initial  policy  likely  to  come  out  as  a  letter  signed  by  Mr.  Young 

•  Actual  policy,  not  guidance 

-  Following  policy  letter 

•  DAG 

•  5000.2 

-  We  are  available  to  support  as  required 

•  AF 

-  SAF/AQR  team  leading  the  charge 

•  Goal  to  issue  guidance  by  1  Jan  08 

•  Policy  memo  signed  my  Ms.  Payton 

•  Put  into  systems  engineering,  AFI  63-1201 

•  Working  with  AQR  to  implement 
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MRA  Deskbook 


•  The  “how-to”  of  MRAs 

•  First  draft  completed  in  March  07 

•  Modeled  after  TRA  Deskbook 

-  Similarities 

•  Achieving  levels  of  readiness  for  risk  reduction 

•  Selection  process  for  CTEs 

-  Differences 

•  Readiness  in  S&T  and  Acquisition  world 

•  Rigorous  assessment  process 

•  Next  draft  based  on  lessons  learned  from 
Reaper  MRA 

-  Dec  07,  Public  releasable 
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Lessons  Learned 


•  MRLs  are  not  a  report  card 

-  MRL  7  might  not  be  good 

-  MRL  3  might  not  be  bad 

•  MRLs  are  a  tool  to  manage  and  mitigate 
manufacturing  risk 

-  A  common  language  used  to  assess 
manufacturing  maturity 

-  Provide  insight  not  oversight 
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Future  Steady  State  dpi 

•  Programs  utilizing  MRLs 

-  Funding  MRL  maturation 

-  Understanding  of  manufacturing  concepts 

•  Use  of  MRLs  in  policy 

-  Program  offices  staffed/trained 

-  Manufacturing  a  key  component  to  MS  reviews 

•  Training 

-  DAU  acts  as  the  primary  government  training  agent 

-  Air  Force  supports  training  updates 
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Additional  Information 


•  MRL  definitions  can  be  found  at  DAU  web  site: 

-  https://acc.dau.mil/CommunityBrowser.aspx?id=18231 

•  Look  for  MR  definitions 

•  Look  for  MR  matrix 

•  Look  for  MRL  tutorial 

•  Google  -  manufacturing  readiness  assessments 
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In  Closing 


•  Using  a  three-pronged  approach  to  implementation 

-  Piloting  and  incorporating  into  various  programs 

-  Training 

-  Policy  insertion 

•  Overall  implementation  progressing 

-  Air  Force 

-  DoD 

•  We  are  still  learning  and  applying  lessons  learned 


Air  Force  is  Leading  DoD-wide  Implementation 
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BACKUP 
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MRL/MRA  Implementation  across  DoD 


ATDs 

ManTech 

SBIR 

Acquisition 

Policy 

Other 

Army 

O 

O 

O 

O 

O 

AIDE 

Program 

Navy/USMC 

O 

o 

O 

o 

o 

Assist  Tool 

Air  Force 

o 

o 

o 

o 

o 

Title  III 

DLA 

N/A 

o 

o 

N/A 

o 

MDA 

N/A 

o 

o 

O 

o 

DARPA 

O 

N/A 

o 

N/A 

o 

O  MRLs 
O  EMRLs 
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